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Abstract Body 


Background / Context: 

Even within a public pre-k program like TN-VPK, there is quite a bit of programmatic variation 
in the amount of gain made by the children within each classroom. Researchers have reported in 
national samples that after accounting for differences in family background, there is still a large 
and significant portion of variation in average school learning that can be explained by 
differences in schools (Borman & Dowling, 2010). Though some variation might be expected, 
given the regional and district differences from one school to the next, one could argue that less 
variation would exist in a monitored, regulated, statewide program almost exclusively serving 
children of low-income families such as TN-VPK. This paper presents evidence on the amount 
of classroom variation in gains across various domains of achievement within the TN-VPK 
program; the data are taken from the RCT design of the TN-VPK evaluation and will explore 
areas where those differences are largest and offer speculations about possible causes of these 
differences. 

Purpose / Objective / Research Question / Focus of Study: 

This paper involves the randomized control trial portion of the larger study which was 
implemented in a limited number of schools with more applicants than seats in the pre-k 
program. Using the RCT, this paper looks at the variability in average classroom-level gains 
during the pre-k year for those children who attended a TN-VPK classroom. 

Setting: 

The general setting of TN-VPK is as described in the previous paper’s abstract. For the RCT 
portion of the larger study, the full randomized sample included children from 111 randomized 
school applicant lists in 28 districts across Tennessee. The consented subsample of that group 
came from 76 applicant lists in 58 schools from 21 districts. Ten of those 58 schools were in the 
West region of the state, 24 were in the Central West region, 12 were in the Central East region, 
and 12 were in the East. 

Population / Participants / Subjects: 

TN-VPK eligibility requirements were outlined in the previous paper abstract. The sample in 
this paper, which is limited to the consented subsample who attended TN-VPK, consists of 773 
students who attended 76 TN-VPK programs; 22% of the students are Black, 16% are Hispanic, 
48% are male, and 84% are Native English speakers. The average age at the time of pretest was 
4.5 years and was 5.1 years at posttest. There was an average of 10.17 study children per 
program (SD=8.1). The 76 programs included 24 urban schools and 52 nonurban schools. 

Intervention / Program / Practice: 

Paper 1 ’s abstract in this proposal describes the nature of TN-VPK classrooms. Because all 
children in the sample for this paper actually attended TN-VPK, we can provide information 
about how much of the intervention was experienced by the students in this paper’s analysis. We 
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defined TN-VPK partieipants as ehildren for whom the available information indieated that they 
attended any TN-VPK program for at least 20 days during the sehool year, whether at the sehool 
that ineluded them on a randomized applieant list or not. This 20-day attendanee requirement 
was identified by the administrator of TN-VPK at the TN State Department of Edueation as the 
minimum number of days required to eonsider a child as having participated in TN-VPK; it 
constitutes one attendance reporting period. For the 773 children in the treatment group of the 
consented subsample, the average number of days attended during the pre-k year was 146.8 with 
a standard deviation of 23.8. 

Research Design: 

The description of the RCT portion of the larger study is provided in the previous paper’s 
abstract. 

Data Collection and Analysis: 

Children were individually assessed using the set of Woodcock Johnson III achievement tests 
that was described in the previous abstract (Letter-Word Identification, Spelling, Understanding 
Directions, Applied Problems, Quantitative Concepts, Passage Comprehension, and Oral 
Comprehension). Data presented here were analyzed through first calculating each individual 
child’s gain by differencing the beginning and end of pre-k scores on each subscale. Then, those 
differences were aggregated to the classroom level for the children who attended VPK. The 
focus of this paper is not just on mean gain but rather on standard deviations and ranges of gain. 

Findings / Results: 

The amount of gain made during the pre-k year in TN-VPK classrooms varied substantially from 
classroom to classroom. Table B1 shows standard deviations and ranges for associated standard 
score mean gains for each of the six subscales. Table B1 clearly demonstrates differences among 
achievement areas in the amount of gain achieved and, most important, in the amount of 
variation in gain seen across classrooms. 

Figure B1 presents the variation by classroom in gains on the Picture Vocabulary subscale, the 
academic area with the smallest gain and among the largest variations. Nearly 45% of the 
schools in the study had average pre-k Picture Vocabulary standard score gains of 0 or less, 
indicating that those programs either did nothing to alter the expected trajectory of gain for those 
children or actually facilitated gain at a lower rate than would be expected given their trajectory 
up to that point. 

Correlations of gains across subscales revealed that though classrooms that tended to make more 
relative gain on one subtest also tended to make relatively more gain on other subtests, the 
strength of the correlations did not suggest that classrooms making the most gains always made 
the most gains regardless of subject matter. Across the 76 schools, correlations between gains on 
any two subtests were never stronger than .365. When we considered only the 49 schools that 
had more than 5 study participants, correlations among subtests were higher but did not exceed 
.497. 
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Conclusions: 


An evaluation of a scaled up prekindergarten program should include information about program 
variability at the local level. It is clear that there is variability in the TN VPK program both 
across classrooms around the state and across achievement domains. While the standard 
deviations were not widely different in gains across areas, the absolute mean gains across areas 
were very different. Picture Vocabulary is of especial concern. Progress in this important area 
was nearly flat for the program as a whole. One conclusion program administrators might draw 
from this finding is that the entire program needs much more attention to vocabulary and 
facilitating language development in the children it serves. Another conclusion could be the 
erroneous idea that vocabulary cannot be facilitated in pre-k classrooms; lack of gain in 
vocabulary is a common finding across prekindergarten and Head Start classrooms. On the other 
hand, close examination of the individual classroom variability presents a somewhat different 
portrait of what’s going on. There are some programs whose children achieved gains in 
vocabulary that are similar to the maximum gains achieved in other areas. These data suggest 
that vocabulary can indeed be developed through participation in pre -kindergarten classrooms, 
but it is obviously not easy. As prekindergarten programs are scaled up, careful attention should 
be made to variations among individual classrooms with early attention to helping high 
performing classrooms provide information that can be applied to the program as a whole. 
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Appendices 
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Appendix B, Tables and Figures 

Table B1 

Descriptive Information on Average Classroom Pre-K Gains in Standard Score Points 


WJ III Subscales 

Minimum 

Maximum 

Mean 

SD 

Literacy 

Letter- Word Identification 

-8.50 

17.50 

5.57 

5.00 

Spelling 

-10.67 

20.00 

4.47 

5.95 

Language 

Oral Comprehension 

-6.50 

15.00 

3.78 

3.91 

Picture Vocabulary 

-19.50 

13.00 

1.02 

5.16 

Math 

Applied Problems 

-8.00 

16.00 

3.23 

4.48 

Quantitative Concepts 

-11.50 

11.83 

3.67 

4.91 
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